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Abstract 

Objective: The objective of this study was to define the minimum academic content required for pediatricians 
to recognize the risk of Icernicterus. 

Methods: A questionnaire was developed on the basis of American Academy of Pediatrics guidelines seeking 
to develop a consensus for pediatricians in training on the theoretical content about neonatal 
hyperbilirubinemia. To validate the instrument, we used the Delphi consensus method. The 14 invited experts 
interviewed, eminent Brazilian researchers of neonatal hyperbilirubinemia, analyzed the questions posed in 
accordance with the literature and validated the instrument 

Findings: An assessment instrument, the Student Questionnaire (SQ), was developed on the basis of 
indicators of risk of neonatal hyperbilirubinemia obtained from the literature. A panel of academic experts, 
composed of the leading researchers of neonatal hyperbilirubinemia in Brazil according to research rankings 
of the Brazilian government's Lattes Platform, was assembled for consensus validation of the assessment 
instrument. Validation of the SQ was achieved after two rounds of the Delphi technique. Finally, the SQ itself 
was validated with the medical education content required for recognition of kernicterus risk. 

Conclusion: The consensus among experts stressed the need to identify the primary epidemiologic risk 
factors for significant hyperbilirubinemia associated with neonatal jaundice and to characterize risk of 
bilirubin encephalopathy according to the literature. The minimum capacity required of physicians in training 
is that they have the insight to consult the reference material specific to each clinical situation in which 
hyperbilirubinemia may be involved. The present study emphasized the need for knowledge of the four 
variables related to management of neonatal jaundice: gestational age, birth weight, infant age, and total 
serum bilirubin. This validated questionnaire can be a useful tool to prepare pediatricians to recognize the 
possibility of bilirubin encephalopathy in neonates and prescribe intervention as necessary. 
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Introduction 

Although the subject of neonatal 
hyperbilirubinemia is part of the curricular 
content of medical schools and graduate Pediatrics 
programs in Brazil, the current literature provides 
no clear proposals on how to practically teach 



physicians in training to manage the icteric 
newborn I*-^]. In 2004, a specialized Committee of 
the American Academy of Pediatrics (AAP] 
published a guideline reviewing the matter [i ^l. 

Due to the metabolic peculiarities of the 
newborn child, hyperbilirubinemia is universally 
found in the first days of life. It can be difficult to 
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manage and, if not identified early and treated 
adequately, can carry a grim prognosis 

Given the current practice of discharging infants 
from maternity care at the 48th hour of life and 
the increased frequency of breastfeeding in 
response to breastfeeding encouragement 
strategies, physiological hyperbilirubinemia- 
which generally arises after the 72nd hour of life, - 
and may be exacerbated by breastfeeding - has 
come to present more intensely, and become a 
frequent reason for hospital readmission after 
maternity discharge P.*-^]. 

The AAP stresses the particular importance of 
looking for risk factors for hyperbilirubinemia or 
kernicterus in full-term or near-term neonates 
about to be discharged 

This study seeks to define specific theoretical 
content geared at guiding practical management of 
neonatal jaundice by producing a theoretical 
assessment instrument composed of practical 
items on the theme of neonatal indirect 
hyperbilirubinemia, with the following research 
question: "What is the minimal theoretical content 
required to give future pediatricians the capacity 
to recognize the risk of kernicterus in jaundiced 
neonates?" 



Advantages of the Delphi method included the 
fact that isolated responses minimized the 
tendency for experts to influence one another, 
while maximizing our ability to discern distinctive 
opinions. Furthermore, the possibility of 
conducting the Delphi rounds via email made this 
a cost-effective and efficient method, saving time 
and money on data input and output PI. 

Ten vignettes on the topic of neonatal 
hyperbilirubinemia were grouped and the 
questionnaire items prepared were validated by 
means of the Delphi method. 

Information was sent to each expert separately. 
Each item was recorded on a report, which was 
resent to each expert for approval or rejection 
until a final consensus was obtained. 
Delphi principles and corresponding 
methodological adaptation: 

a} "Which indicators were selected for the 
questions to be formulated to explore the 
matter?" - questionnaire items 

b] "Who are the experts, and where are they?" - 
selected experts 

c] "Which technique will be applied?" - 
systematic Delphi stages 

d] "What are the expected results?" - instrument 
validation. 



Subjects and Methods 

This study was structured as a descriptive survey 
leading to the development of an instrument that 
could answer its research question. The Delphi 
consensus method for expert-based questionnaire 
validation was used. In designing the 
questionnaire, we used the current literature to 
define key points for practice in the management 
of neonatal jaundice before discharge. These 
indicators were also tied in with the 2004 AAP 
guidelines P.8,12], 

One of the limitations of this method as used in 
our study is that experts were not included in the 
primary definition of these questions, only in their 
later discussion. We believed it would have been 
inappropriate to burden these busy experts with 
such an arduous and time-consuming task. They 
did, however, take part in assessing the form, 
content, and detail of each of the items presented. 



Findings 

The ten items of the Student Questionnaire (SQ] 
were as follows: 

1] Epidemiological indicators of worsening 
hyperbilirubinemia; 2] predictors of risk of 
bilirubin encephalopathy; 3} visual clinical 
recognition of jaundice and its hazards; 4] 
correlation between clinical indicators of 
encephalopathy and progression to kernicterus; 5] 
laboratory assessment of pathological neonatal 
jaundice; 6] management of indirect hyper- 
bilirubinemia; 8] management of neonatal 
jaundice detected before the 48th hour of life; 9] 
diagnostic definition of neonatal jaundice at 
review (after discharge}; 10] minimum systematic 
approach for recognition of kernicterus. 
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The Experts 

Our pool of experts was composed of National 
Council for Scientific and Technological 
Development (CNPq] fellows conducting research 
on neonatal indirect hyperbilirubinemia who 
agreed to take part in the study. The criteria for 
expert selection used the Lattes Platform, an 
online database maintained by the Brazilian 
government and containing information on 
researchers and research institutions working in 
science, technology, and innovation in the country. 
The database was searched for medical school 
professors who had authored articles with the 
descriptors "neonatal jaundice", "newborn hyper- 
bilirubinemia", or "neonatal indirect hyper- 
bilirubinemia", published in indexed journals 
between 2002 and 2006. We invited the top 30 
such authors [as of 2007] to take part in the study. 

Of these, the first 15 to agree were selected to 
make up the expert panel; 14 remained on the 
panel until the end of the study. 

Delphi Stages 

The stages of SQ development, from concept to 
validation, were: 

1] Production of questionnaire [target audience: 
pediatricians in training]; 2] questionnaire sent to 
experts who agreed to take part; 3] consensus 
analysis of the first expert responses received; 4] 
modification of SQ and pilot study to assess item 
clarity; 5] second [modified] SQ sent to experts; 6] 
consensus report, concluding that the SQ was 
validated by the expert panel. 

Instrument Validation 

The target demographic of the measurement 
instrument comprised future pediatricians who 
were finishing medical school or had graduated 
from medical school no more than 6 months 
before responding to the questionnaire. 

The SQ sent to the 14 selected experts for 
validation comprised ten items, or questions, of 
theoretical and practical relevance to "neonatal 
hyperbilirubinemia in the first days of life". All 12 
items of the Expert Questionnaire or EQ [with 
item 8 subdivided into two, for a total of 13 
questions] were designed to enable validation of 
the 10-item SQ. Experts were asked to measure 
the relevance and importance of proposed 
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content; give their opinion as to whether content 
should be kept or changed, suggesting 
improvements as necessary; and give their 
opinion on whether additional relevant items 
should be included, or redundant items removed. 

Expert validation of each SQ question focused 
on reviewing 

1] instructions; 2] clarity; 3] subjectivity; 4] bias; 
5] redundancy; 6] omission; 7] success [i.e., the 
potential for successful responses]; 8] compre- 
hensiveness in terms of a] hyperbilirubinemia and 
b] kernicterus; 9] overall comprehensiveness; 10] 
item weights; 11] relevance of content; 12] 
format/presentation [Fig. 1]. The validated 
consensus on content, presentation, and 
qualification was pooled to become the overall 
consensus I^l, obtained by means of the EQ [Table 
1], which was used to define each SQ item as 
indispensable, important, or expendable and 
[based on presentation] whether each SQ item 
should be kept, modified, or removed. 

On the basis of overall qualification of means 
and standard deviations of Likert-type 
assessments, each SQ item was defined as 
complete/perfect, fair, poor, or unacceptable. All 
comments made by experts during 
epidemiological assessment and validation of the 
SQ were taken into account. The literature 
framework for the 10 SQ items was combined with 
the comments made by the 14 experts to validate 
the instrument. Analysis enabled description of 
the mean [±standard deviation] as the likelihood 
of the statistical outcome. The consensus for final 
validation was thus defined. 

Student Questionnaire (SQ) 

The Delphi method proved effective in 
constructing a consensus expert opinion on the SQ 
after two rounds of study. 



Discussion 

Overall, the structure of the pilot version of the SQ 
was modified on the basis of the consensus expert 
decision that, when faced with difficulties in 
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Table 1: Consensus of expert questions [EQ] for validation of the Student Questionnaire [SQ) 



CONSENSUS 


1 




1 


d 


e 


ii 


7 


VALIDITY 


1 




J 






O 


/ 


Formative 
content* 
mean Qll- EQ 


expendable/ 
no 

importance 


very little 
importance 


little 
importance 


important 


very 
important 


extremely 
important 


indispensable 
/ of greatest 
importance 


Formulation 
or 

presentation t 
mean Q12- EQ 


remove 


modify 
nearly 
everything 


modify 
consi- 
derably 


modify 


modify 
little 


modify 
very little 


Keep 


Qualification'!' 
MoM, 12/13 SQ 
Qs 


unacceptable 


very poor 


Poor 


not very 
adequate 


good 


very 
good 


complete/ 
perfect 


Overall mean 
for the 3 
consensuses 


unacceptable 


very poor 


Poor 


not very 
adequate 


good 


very 
good 


complete/ 
perfect 


Adequacy* 
QMoM+MF>60% 


not feasible 


very poor 


Poor 


not very 
adequate 


good 


very 
good 


complete/ 
perfect 


Rtg. Variation * 
FM var. EQ Q. 
Rtg. 


complete/ 
perfect 


very good 


Good 


not very 
adequate 


poor 


very poor 


Nonexistent 



*mean of 14 experts' opinions regarding content, 10 SQ items/11 EQ items; 
tmean of 14 experts' opinions regarding format, 10 SQ items/12 EQ items; 

4:mean of means [MoM] for 14 experts' opinions regarding the 10 SQ items in the 12/13 EQ items; 

4:final mean for numbers corresponding to the cumulative percentage >50% in the 10 SQ questions for each of the 12/13 EQ 
questions; 

+ mean of means (MoM) for all opinions of all 14 experts regarding the 10 SQ items per response to the 12/13 EQ items. 
#final mean or rating variation in EQ expert rating for the 10 SQ items. Here, the scale is inversely proportional: the fewer points 
(denoting less variation], the better the consensus. 



management of neonatal jaundice, physicians 
should consult the relevant literature. This is 
particularly important in the study sample, which 
consisted of inexperienced young physicians or 
medical students, who still lack the expertise and 



discernment required to carry out risk assessment 
on the basis of clinical observation alone. 

The present study took on a more instructional 
nature through the addition of reference material 
for assessment of hyperbilirubinemia risk, 
including hour-specific nomograms for hyper- 
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0 I , • , , , , , . , , , : , 

Instruct, Clarity Subjecty. BiB3 Redund. Indus. Success Measure Assess Global Rating Content Form 

bilirubin kern. assessmt 

Criteria 

Flg.l: Expert validation ofthe 10-item Student Questionnaire (SQ); means and standard deviations of the 12/13- 

item Expert Questionnaire (EQ] 
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Fig. 2: Hour-specific nomogram of mean TSB levels (mg/dL] indicating high-intensity 
phototherapy in newborns with a gestational age >35 weeks and birth weight >2 kg 



bilirubinemia, to the questionnaire. Of note, TSB, 
gestational age (GA], birth weight [BW], and age 
must always be recorded and presented for 
nomogram analysis. 

Some unanimous expert suggestions were used 
to enrich the vignettes used in the questionnaire, 
but the material provided as background for 
contextualization of answers (Fig 2, 3 and Box 1] 
was based exclusively on AAP guidelines, which 
were, in turn, based on robust epidemiological 
data [15]. 



An example of risk characterization based on the 
SQ items and not on the references used to build 
the SQ was the mention of Kramer's rule. Although 
the caudal progression of clinical jaundice is not a 
proven prognostic indicator of severity, it was 
suggested during the expert validation process. 
Greater emphasis was afforded to the relevance of 
the difficulty of clinically observing jaundice 
during phototherapy, particularly in patients with 
darker skin and/or in the absence of natural 
lighting [16]. 
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Fig. 3: Hour-specific nomogram of mean TSB levels [mg/dL] indicating exchange transfusion 
in newborns with a gestational age >35 weeks and birth weight >2 kg. 
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BOX 1: Epidemiological risk factors for worsening of significant hyperbilirubinemia in neonates with gestational 

age >35 weeks and birth weight >2 kg 

GREATER RISK: Total serum bilirubin (TSB] level in the high risk area of the hour-specific nomogram prior to 

discharge; gestational age between 35 and 36 weeks; jaundice in the first 24 hours of life; Rh 
incompatibility, positive Coombs test and other hemolytic disorders, such as ABO 
incompatibility and reduced glucose-6-phosphate dehydrogenase levels; siblings who 
required phototherapy at birth; cephalohematoma or ecchymosis; exclusive breastfeeding 
with supraphysiological weight loss, in excess of 10% in the first week of life; Asian ethnicity. 

INTERMEDIATE TSB level in the intermediate risk area of the hour-specific nomogram prior to discharge; 

RISK: gestational age between 37 and 38 weeks; macrosomic child born to diabetic mother; 

maternal age >25 years; male gender. 

MINIMAL RISK: TSB level in the low risk area of the hour-specific nomogram prior to discharge; gestational 

age >40 weeks; exclusively formula-fed; discharged after 72nd hour of hfe. 



Experts noted the difficulty of assessing the risk 
of hyperbilirubinemia by visual observation of 
jaundice even when carried out by experienced 
neonatologists. In addition to the factors described 
in the AAP material, experts mentioned the need 
for careful assessment of the following neurotoxic 
factors for kernicterus: administration of drugs 
that compete with albumin binding of bilirubin 
[vitamin K, sulfas, ceftriaxone]; and intracranial 
bleeding with increased cerebral blood flow. We 
note that the potential for adverse effects of high- 
dose synthetic vitamin K therapy was addressed 
years ago, and no longer poses any risk, as long as 
appropriate dosages are used [i^.is] 

The experts also stressed the importance of 
quantitating glucose-6-phosphate dehydrogenase 
[G6PD] whenever prolonged exposure to 
adequate phototherapy is unable to effect a 
reduction in hyperbilirubinemia, regardless of 
ethnicity or family history of G6PD deficiency 

[3,7,13]_ 



Another point of emphasis was that "intensive 
phototherapy", defined as the use of high levels of 
irradiance - in the 430 to 490 nm range, 
30^W/cm2/nm or higher - should be measured 
directly at the bassinet by a radiometer whenever 
indicated [3.1*1. 

Expert suggestions were used to expand and 
adapt the SQ vignettes to the Brazilian reality. 

For instance, in adapting the AAP guidelines to 
the Brazilian reality, we removed the need for 
notification of kernicterus to the CDC, as Brazil has 
no equivalent organization to receive this 
information. We also disregarded the possibility of 
transcutaneous bilirubin measurement, which 
would only be possible with bilirubinemia below 
15 mg/dL [1^1. The necessary equipment is ususUy 
not available in Brazilian care facilities. 

Furthermore, the device mentioned in the AAP 
guidelines for measurement of ETCOC as it relates 
to hyperbilirubinemia is extremely costly, and 
used only in clinical research, thus making it a low 
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Fig. 4: Mean of means, consensus opinions of the 14 experts on the 10 SQ items 
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priority for acquisition in Brazil. 

Another consensus comment was that, in the 
etiologic diagnosis of jaundice, one must stress 
that variations from birth weight upon follow-up 
exam generally coincide with a physiological 
weight reduction, which may be confused with low 
or inadequate infant feeding. 

The last validated SQ question recommended a 
minimum response standard, focusing on 
diagnostic and therapeutic strategies applicable to 
healthcare professionals and to the caregivers of 
newborn infants, meant to address the 
repercussions of jaundice as early as the first days 
of life, using data from prenatal care and including 
intervention whenever necessary [i8.2o,2i]_ 

Our decision to recruit experts with teaching 
experience and applying their suggestions to a 
questionnaire administered to a sample of young 
physicians and medical students appears to have 
reached its primary objective: to develop an 
accurate, validated instrument for determination 
of the minimum theoretical content required to 
teach kernicterus risk assessment to physicians in 
training. 

We humbly thank the 14 investigators who 
assisted in the construction of this instrument in 
their capacity as Brazil's foremost experts in 
neonatal hyperbilirubinemia. 



Conclusion 

In conclusion, the minimum theoretical content 
that must be part of medical education programs if 
pediatricians are to identify and address neonatal 
jaundice in an appropriate manner is mostly tied 
to building awareness of the epidemiological risk 
factors for indirect hyperbilirubinemia and of the 
potential risk characteristics for bilirubin 
encephalopathy, in light of the specific literature 
guidelines for clinical conduct analysis of each 
case. 

The guidelines and rules for reference are those 
found in the literature, and must be made 
available in services that provide neonatal care. 
The minimum primary care capacity physicians 
must have is the discernment to consult reference 
materials specific to each chnical situation where 
hyperbilirubinemia may be involved. 



This study places particular emphasis on 
knowledge of the four variables that should guide 
management of neonatal jaundice: gestational age 
at birth, birth weight, extrauterine age, and total 
serum bilirubin. 

This validated questionnaire should prove a 
useful instrument, preparing pediatricians to 
recognize the possibility of bilirubin 
encephalopathy in susceptible neonates and to 
provide timely intervention whenever necessary. 
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